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ABSTRACT 

Research has shown a strong pilot preference for 
predictive information of aircraft system status in the 
flight deck. However, changes in pilot behavior 
associated with using this predictive information have 
not been ascertained. The study described here 
quantified these changes using three types of predictive 
information (none, whether a parameter was changing 
abnormally, and the time for a parameter to reach an 
alert range) and three initial time intervals until a 
parameter alert range was reached (ITIs) (1 minute, 5 
minutes, and 15 minutes). With predictive information, 
subjects accomplished most of their tasks before an 
alert occurred. Subjects organized the time they did 
their tasks by locus-of-control with no predictive 
information and for the 1 -minute ITI, and by aviate- 
navigate-communicate for the time for a parameter to 
reach an alert range and the 15 -minute conditions. 
Overall, predictive information and the longer ITIs 
moved subjects to performing tasks before the alert 
actually occurred and had them more mission oriented 
as indicated by their tasks grouping of aviate-navigate- 
communicate. 

INTRODUCTION 

Documented instances exist where some type of early 
notification to the flight crew of a system parameter 
deviation could have prevented or lessened the 
consequences of an aircraft’s system failure [1], [2], 
This early notification may improve decision making 
by allowing the flight crew to be more informed, thus 
increasing the safety of the flight [3]-[5], Current 
research has shown that predicting system failures bring 
potential benefits for increased safety of flight [6]-[9]. 

Research has indicated a strong pilot preference for 
predictive information [6], Other results garnered from 
this study have demonstrated that pilots do use the 
predictive information to affect the alert and its 
consequences [8], For instance, with predictive 
information (in the form of knowing that a parameter 
was moving abnormally or of knowing the time to an 
alert), pilots retrieved checklists, descended, diverted, 
and declared an emergency earlier [8], These results 
also indicated that the longer the time pilots had until a 
parameter reached an alert range, the earlier they 
performed various tasks [8], 


The same research, though, also indicated that pilot 
perceived workload increased slightly with the 
predictive information [9], In addition, pilot perceived 
situation awareness decreased with predictive 

information but increased with increasing initial time 
intervals until a parameter alert range was reached 
(ITIs). This, in addition to pilots performing certain 
tasks earlier with increasing ITI, reemphasized that the 
changes in pilot behavior should also be analyzed. 

The possible changes in pilot behavior as defined by 
the order (absolute and relative) pilots would perform 
certain tasks and how this order affected their stress 
levels had not yet been fully analyzed. Currently, flight 
crews have defined procedures and tasks they must 
follow when a parameter reaches an alert range but no 
such procedures exist regarding a parameter moving 
towards an alert range. Therefore, it was of interest to 
determine if and how predictive information might 
affect pilot behavior. Thus, this paper presents the 
results from an investigation of the behavior of pilots 
exposed to various levels of predictive information. 

Objectives 

This experiment was conducted to determine the 
effects predictive information would have on pilot 
behavior in an operational setting during non-normal 
system events. Pilot behavior was defined as the tasks 
pilots did and when they did these tasks. The tasks 
were: when the pilot noticed the failure, accessed the 
appropriate checklist, took action to affect the alert, 
turned the plane off-track, diverted, declared an 
emergency, called dispatch, talked to the flight 
attendant, and obtained weather information at the 
diversion airfield. 

Experimental Variables 

Of the four experimental variables, two were directly 
manipulated: the predictive information available and 
the ITI. The predictive information available, a 
between-subject variable, was one of three types: (1) 
none (baseline), (2) whether a parameter was increasing 
or decreasing abnormally (direction), or (3) the time 
remaining to a parameter alert range (countdown). The 
second variable was the ITI, a within-subject variable, 
and it had four levels: (1) 1 minute, (2) 5 minutes, (3) 
15 minutes, and (4) ETA+45 minutes (Estimated Time 



to Arrival). Baseline predictive information and 
ETA+45 minutes ITI, which place the alert beyond the 
end of the flight, were control conditions. The third 
experimental variable was the four independent faults 
each subject encountered, and was partially controlled 
in that the parameter would degrade in a regulated 
manner. The last experimental variable, task, was 
calculated based on when subjects performed certain 
tasks, such as diverting. 

Predictive Information Available . In the baseline 
condition, no predictive information was available. 
Thus, when a parameter reached an alert range, the 
subjects saw the typical alert message (e.g., CABIN 
ALT) with the accompanying aural alert (table 1). In 
the other two conditions, direction and countdown, a 
text message on the alerting system screen notified 
subjects that a parameter was moving towards an alert 
range once a failure occurred. For the direction 
condition, subjects were told that a parameter was 
increasing or decreasing abnormally (table 1). For the 
countdown condition, subjects were told when a 
parameter would reach an alert range for the given 
aircraft state (table 1). The onset of an alert was 
updated in increments of whole minutes if the time 
remaining was greater than 1 minute. If the time to an 
alert was less than 60 sec, the message updated for 
every 15-sec change in the onset of an alert. 


Table 1 - Examples of Predictive Information 


Condition 

Predictive 

Information 

Alert 

Information 

Baseline 

none 

“CABIN ALT” 

Direction 

“CABIN ALT INC” 

“CABIN ALT” 

Countdown 

“CABIN ALT 7MIN” 

“CABIN ALT” 


In all cases, the predictive information presented to 
subjects was always correct and accurate. The 
predictive information had an alert category of 
advisory. Furthermore, parameters increased or 
decreased at a constant rate dependent on the state of 
the aircraft. Lastly, in the direction and countdown 
conditions, the related standard alert information 
message replaced the predictive information message 
when the parameter reached an alert range. 

ITIs . Each subject saw four ITIs (the time interval to 
an alert once a failure began): (1) 1 minute, (2) 5 
minutes, (3) 15 minutes, and (4) ETA+45 minutes. The 
configuration of the aircraft affected the actual time to 
an alert; for example, throttling back the affected 
engine during the scenario with the EGT (Exhaust Gas 
Temperature) increase delayed the onset of an alert. 


Faults . All four data runs, or scenarios, and the 
training run included a fault in which a parameter 
would eventually reach an alert range if the subject 
took no action. The faults were (1) cabin altitude 
increase, (2) forward cargo overheat, (3) EGT increase, 
and (4) oil quantity decrease. The training run had an 
avionics overheat. All failures were designed to behave 
as realistically as possible [10]-[13] and are described 
below. 

For the scenario with the cabin altitude increase, the 
cabin altitude increased to the airplane altitude. The 
outflow valve, if checked, was fully closed once the 
failure started. Although the increase could not be 
controlled through the environmental system, the cabin 
altitude warning would not be reached if the subject 
descended below 10,000 ft mean sea level and if he had 
at least 4 IT minutes until the alert range was to be 
reached — the time needed to descend from the initial 
altitude of 37,000 ft to 10,000 ft. 

In another data run, the forward cargo hold, initially 
set-up for carrying animals, had a temperature increase. 
If the subject changed to the cargo mode, the 
temperature increase would slow due to the lower 
temperature setting. Also, if he discharged the forward 
cargo fire bottle before the alert range was reached, the 
forward cargo temperature would never reach the alert 
range. If the subject discharged the forward cargo fire 
bottle after the fire warning, as the forward cargo fire 
checklist instructs him to do, the temperature would 
drop below the alert range. 

During the scenario with the EGT increase, the EGT 
rose steadily and if it reached the alert range, the 
subject would have to follow the engine 
failure/shutdown procedure. The subject could slow the 
increase if he throttled back the engine with the 
increasing EGT or stopped if he shut down the affected 
engine. If the subject restarted the engine, the EGT 
would again increase until it reached the alert range. 

Finally, one scenario resulted in an oil quantity 
decrease due to loss of oil. The decreasing oil quantity 
drove the oil pressure down and the oil pressure 
triggered the alert once it reached an alert range. The 
only way to decrease the rate of oil loss was to shut 
down the affected engine. 

For the avionics overheat training run, changing the 
avionics mode to override from its initial position of 
normal decreased the rate of temperature increase. 
Furthermore, by disconnecting bus 3, the temperature 
would stay below the warning limit. Thus, the load on 
bus 3 was the primary cause of the overheat. 




EXPERIMENT DESIGN 


Tasks . Several tasks the subjects could take that 
would affect the onset of an alert and the gravity of the 
failure were of interest. These tasks were when they (1) 
accessed appropriate checklists, “checklist,” (2) took 
actions to affect the time to an alert, “action,” (3) 
diverted, “divert,” (4) descended, “descend,” (5) 
obtained weather information at the diversion airfields, 
“weather,” (6) turned off-path, “turn,” (7) declared an 
emergency, “emergency,” (8) talked to the head flight 
attendant, “flight att,” and (9) called dispatch, 
“dispatch.” The time when (10) the warning occurred, 
“warning,” and (11) the failure started compared to 
when a warning occurred or would have occurred, “init 
alert,” were also recorded. 

All failures had checklists associated with them. 
Thus, if a parameter reached an alert, the subject should 
follow the checklist. A subject could access the 
checklists before the parameter reached an alert range if 
he so desired. 

The specific actions considered as trying to affect the 
onset of an alert depended on the scenario. These 
actions were checking the outflow valve position for 
the cabin altitude increase failure, changing the forward 
cargo compartment mode from animal to cargo or 
discharging a forward cargo compartment fire bottle for 
the forward cargo fire scenario, and throttling back or 
shutting down the affected engine for the EGT increase 
and oil quantity decrease scenarios. 

Since the flight was under ETOPS (Extended Twin- 
engine OPerationS) rules, subjects had to divert for the 
oil quantity decrease and EGT increase failures when 
they shut down an engine. The ETOPS rules do not 
specify a diversion is necessary with cabin pressure 
loss, but for fuel efficiency reasons and passenger 
comfort, the logical choice would be to divert. If a 
subject had a forward cargo fire warning, he would be 
required to divert under ETOPS rules. If a subject 
discharged the fire bottle before the fire warning, thus 
preventing the temperature from increasing into the 
alert range, he did not technically have to divert, but 
prudence recommended diverting anyhow because of 
the strong possibility of fire. 

Three of the faults required subjects to descend: (1) 
the cabin altitude increase, and after engine shutdown 
for both (2) the EGT increase and (3) the oil quantity 
decrease. Also, checking weather at the diversion 
airport, turning off path, declaring an emergency, 
telling the flight attendant the status of the flight, and 
calling dispatch to let the company know the current 
situation was not explicitly required but was considered 
good airmanship. 


Subjects 

Twelve glass-cockpit airline pilots familiar with 
ETOPS rules participated as subjects. Seven were 
currently first officers, the remaining five were 
captains. The average age was 48 years old and the 
average commercial airline flight experience was 16 
years. 

Test Design 

This experiment was run in the Advanced Civil 
Transport Simulator at the NASA Langley Research 
Center. This simulator had flight performance 
characteristics similar to a Boeing 757. The flight deck 
resembled a Boeing 747-400 or MD-11. The subject 
acted as captain, pilot-not-flying. A confederate first 
officer (F/O) was pilot flying and he was well versed in 
the operation of the simulator. A confederate air traffic 
controller (ATC) and company dispatch operator 
provided the necessary coordination with the ground. 

The flight was from Dulles airport to Charles de 
Gaulle airport with a 60-minute ETOPS rule; i.e., the 
plane was never more than 60 minutes from an alternate 
airfield. The 60-minute rule was used in order to have 
several PETs (Point of Equal Time); i.e., the point 
where the plane was 60 minutes from any suitable 
alternate airport. The scenarios were set up such that 
each segment of flight started before a PET; thus, this 
experiment only included the cruise phase of flight. If 
the configuration of the aircraft did not change dining 
the fault, the affected parameter would reach an alert 
range a few minutes before the aircraft intersected the 
PET except in the ETA+45 minute condition. 

Any materials and information the subject needed 
were provided to him. Plotting charts, landing plates, a 
dispatch weather briefing, and a flight plan were 
available in paper form. Checklists were electronic and 
mimicked the Boeing model of the quick reference 
handbook [14], Voice communication was used for 
ATC and dispatch. Both ATC and dispatch were able 
to supply current weather information at any of the 
diversion airfields. Basically, the weather at all 
diversion airfields was acceptable for landing — drizzle 
with a ceiling around 1,000 ft and visibility 
approximately IV 2 miles with winds at no more than 10 
knots. ATC also reasonably expedited (within 15 sec) 
any requests subjects had regarding course changes. 
The confederate F/O was able to answer operational 
questions from the subject; i.e., he supplied all the 
operational information normally found in the aircraft 
manual. Lastly, subjects made any passenger 



announcements or held conferences with the head flight 
attendant, or purser, to the experimenter sitting in the 
back of the simulator. 

As mentioned earlier, the faults, the ITIs, and tasks 
were within-subject variables while the predictive 
information was between subjects. Since subjects could 
only see each failure once, each subject had four data 
runs in addition to a training run. Thus, all subjects saw 
each of the four faults once and each of the four initial 
times to an alert once with only one of the three types 
of predictive information. 

Dependent Measures 

The dependent measure was when subjects 
performed the various tasks listed above. 

Procedure 


A multidimensional scaling using SPSS®, Statistical 
Product and Service Solutions, was done calculating an 
Euclidean distance from the normalized time ratio data 
[15], SPSS® was also used to do a hierarchical cluster 
analysis again using the normalized time ratio data. 

RESULTS 

Order of Tasks 

The order subjects accomplished the tasks are similar 
except for some transpositions (figs. 1 and 2). With 
predictive information (fig. 1), subjects performed the 
vast majority of tasks earlier with the direction 
predictive information than in the baseline condition, 
and even earlier with the countdown predictive 
information. Subjects also did tasks earlier the greater 
the ITI (fig. 2). In fact, for the 15-minute ITI, subjects 
accomplished everything before the alert. 


When a subject first arrived, he received an overview 
on this experiment. After this introduction, the 
confederate F/O gave a detailed description of the 
simulator and its operation, and the flight plan to the 
subject before the training run started. The training run 
included the avionics overheat fault 15 minutes into the 
flight. The ITI was 5 minutes given the initial aircraft 
configuration. No data were recorded during training. 

A short break was taken after the training run and 
before data run 1. An hour lunch break followed the 
first data run. After lunch, the subject completed data 
runs 2 through 4. Each data run took approximately 30 
minutes. At the end of each data run, the subject was 
asked about the failure, his actions, and his workload. 
The presentation order of predictive information and 
initial time to an alert were counterbalanced while 
scenario order was only partially balanced due to the 
number of subjects. 

Data Analysis 

All times were normalized in order to extricate the 
fact that different ITIs occurred during the flight. If the 
times were not normalized, the data clustered around 
four discrete categories dependent on the initial time to 
a parameter alert range. The normalized time ratio was 

. . time at which X occurred 

normalized time ratio = . 

actual time to alert 

Times were taken from failure start. The actual time to 
alert was when the alert truly occurred or would have 
occurred had the subject not done something to prevent 
it such as shut down an engine. Subjects were not 
penalized in the data analysis if they did not perform a 
particular task. 
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Figure 1 - Time Tasks Performed by Predictive 
Information 
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Multidimensional Scaling 


The multidimensional scaling for the normalized time 
ratio data by predictive information and 1-, 5-, and 15- 
minutes ITIs had one dimension but the one dimension 





differed depending on the particular condition (figs. 3 
and 4). Not enough data were present for the ETA+45 
minute condition. The dimension changed with 
increasing predictive information and increasing ITI. 
For the baseline and 1 -minute ITI, the dimension was 
locus-of-control. The locus-of-control was defined by 
how much control the subject was able to exert over the 
task. At the other end of the spectrum, countdown 
predictive information and 15-minutes ITI, the 
dimension entailed aviate-navigate-communicate. For 
the direction and 5-minute conditions, the dimension 
was muddled because it was transitioning from locus- 
of-control to aviate-navigate-communicate. 



Predictive Information 

Figure 3 - Clustering by Predictive Information 



Initial Time Interval to Parameter Alert Range 
Figure 4 - Clustering by ITIs 

Cluster Analysis 

A cluster analysis for the normalized time ratio data 
was done for each predictive information condition and 
for each ITI except for ETA+45 minutes since not 
enough data were present for that condition. Figures 3 


and 4 show the cluster analysis. The clustering for 
baseline predictive information and 1 -minute ITI are 
the same. The clusters were interpreted to be inner-loop 
locus-of-control to outer-loop locus-of-control. For the 
countdown predictive information and for the 15- 
minute ITI, the groups are aviate, navigate, and 
communicate. As mentioned with the multidimensional 
scaling above, the clusters for the direction and 5- 
minute conditions were muddled since the dimension is 
changing from locus-of-control to aviate-navigate- 
communicate. 

DISCUSSION 

To explore the effects predictive information would 
have on pilot behavior, a simulator experiment tested 
three types of predictive information and four ITIs. The 
three types of predictive information were (1) baseline, 
(2) direction, and (3) countdown, and the four ITIs 
were (1) 1 minute, (2) 5 minutes, (3) 15 minutes, and 
(4) ETA+45 minutes. 

As the amount of predictive information increased 
(baseline to direction to countdown) or the ITI 
increased, subjects accomplished most of their tasks 
before the actual alert occurred. Note that the time to 
task and checklist access in the baseline condition may 
have been artificially low because subjects were primed 
for a failure. Thus, they may have been more diligent 
scanning the instruments looking for deviations. 

Subjects organized the time they did their tasks by 
locus-of-control for the baseline predictive information 
and 1 -minute ITI, and by aviate-navigate-communicate 
for the countdown and 15 -minute conditions. For the 
other two conditions (direction and 5-minutes), the 
dimension was in transition from locus-of-control to 
aviate-navigate-communicate. For the 15-minute ITI, 
the ordering of the groups was aviate, communicate, 
navigate. As compared to the countdown condition, the 
15-minute condition gave subjects enough time to 
broadcast their intentions before having to carry them 
out; thus, communicate and navigate switched places. 
Notice that the baseline, and 1- and 5-minute conditions 
have the same ordering although the exact values are 
different. The similarity of the baseline and 1 -minute 
ITI suggests that the 1 -minute ITI is not enough of a 
forewarning to be of benefit. Furthermore, the 
similarities of the 5-minute ITI plus its transitional 
groupings to the baseline condition suggests that a 5 
minute forewarning gives the subject just enough time 
to affect the failure and the onset of an alert. This is 
further substantiated by the vast majority of tasks being 
completed before the alert for the 5- and 15 -minutes 
ITI. 





These results suggest that with the foreknowledge of 
a possible problem and with enough of a forewarning 
(>5 minutes), subjects were able to plan their tasks 
thinking more of the overall mission, aviate-navigate- 
communicate. This movement also corresponded to 
previous data analysis which found that situation 
awareness increased with increasing ITI and that 
subjects with predictive information focused more on 
the mission (aviate-navigate-communicate) rather than 
the spatial aspect (locus-of-control) of the aircraft, 
which is appropriate since subjects as pilots-not-flying 
should have been more concerned about the overall 
mission [9]. 

CONCLUSIONS 

Overall, predictive information and the longer 
prediction times moved subjects to performing tasks 
before the alert actually occurred and had them move to 
a more mission oriented center indicated by their tasks 
becoming more strategic and grouping by aviate- 
na vig ate -communicate . 

Other aspects, though, must be investigated before 
the full usefulness of predictive information can be 
understood. Further research into the optimal prediction 
time, acceptable false alarm rate, and accuracy of the 
predictive information must be done. Also, it would be 
of benefit to ascertain how useful the information 
would be when pilots are not primed for a failure. On 
the more operational side, the ability to estimate the 
time to an alert with the false alarm rate and accuracy 
required by the pilots needs to be investigated before 
procedures are developed using predictive information. 
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